MDEV-39791: Handle count aggregate optimization for replay purpose by bsrikanth-mariadb · Pull Request #5145 · MariaDB/server

bsrikanth-mariadb · 2026-05-29T07:07:20Z

During replay, when get_exact_record_count() is invoked from opt_sum_query(),
hook the recorded table rows from the parsed context for each table,
during the calculation of count.

gemini-code-assist

Code Review

This pull request introduces support for recording and replaying count aggregate contexts (count_agg) within the optimizer context store/replay framework. While the implementation successfully integrates the new structures and serialization routines, several critical issues must be addressed. Most notably, there is a type confusion vulnerability in opt_sum.cc where a cast to Item_field* is performed without verifying the item type, which could lead to server crashes. Additionally, there are bugs involving the serialization of a pointer instead of the actual count value, incorrect string appending of a ulong counter, potential counter desynchronization during OOM events, and swapped error messages in the JSON parsing routines.

spetrunia · 2026-06-03T11:42:13Z

MariaDB [test]> set optimizer_record_context=1;
Query OK, 0 rows affected (0.001 sec)

MariaDB [test]> create table t10 (a int not null) engine=myisam;
Query OK, 0 rows affected (0.014 sec)

MariaDB [test]> insert into t10 select seq from seq_1_to_3;
Query OK, 3 rows affected (0.004 sec)
Records: 3  Duplicates: 0  Warnings: 0

MariaDB [test]> explain select count(*) from t10;
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
| id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | Extra                        |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
|    1 | SIMPLE      | NULL  | NULL | NULL          | NULL | NULL    | NULL | NULL | Select tables optimized away |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
1 row in set (0.001 sec)

^ this is not caught by capture at all.
This EXPLAIN output means that the query did use get_exact_record_count...

To check that I do have your patch, etc:

MariaDB [test]> explain select count(a) from t10;

Now, this does hits a breakpoint in Optimizer_context_recorder::record_count_agg, as expected.

spetrunia · 2026-06-03T11:43:21Z

count_agg
record_count_agg
infuse_count_agg

This is a bad name. Please change to exact_row_count .

spetrunia

Please fix the above

bsrikanth-mariadb · 2026-06-04T04:44:23Z

count_agg
record_count_agg
infuse_count_agg
This is a bad name. Please change to exact_row_count .

sure. changed it.

bsrikanth-mariadb · 2026-06-04T04:47:06Z

MariaDB [test]> set optimizer_record_context=1;
Query OK, 0 rows affected (0.001 sec)

MariaDB [test]> create table t10 (a int not null) engine=myisam;
Query OK, 0 rows affected (0.014 sec)

MariaDB [test]> insert into t10 select seq from seq_1_to_3;
Query OK, 3 rows affected (0.004 sec)
Records: 3  Duplicates: 0  Warnings: 0

MariaDB [test]> explain select count(*) from t10;
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
| id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | Extra                        |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
|    1 | SIMPLE      | NULL  | NULL | NULL          | NULL | NULL    | NULL | NULL | Select tables optimized away |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
1 row in set (0.001 sec)

^ this is not caught by capture at all. This EXPLAIN output means that the query did use get_exact_record_count...

To check that I do have your patch, etc:

MariaDB [test]> explain select count(a) from t10;

Now, this does hits a breakpoint in Optimizer_context_recorder::record_count_agg, as expected.

Yes, count(*) would also be hitting the opt_sum_query() call, and we could get the count. However, I see one problem. How could we get the table information if the expression inside the count() is not a field_type? For count(a) I could get table info from field a.

spetrunia · 2026-06-04T08:01:09Z

However, I see one problem. How could we get the table information if the expression inside the count() is not a field_type? For count(a) I could get table info from field a.

The optimizer already handles all this.

create table t10 (a int, b int not null) engine=myisam;
insert into t10 select seq,seq from seq_1_to_10;
create table t11 like t10;
insert into t11 select seq, seq from seq_1_to_3;

If the expression is not NULLable, it doesn't matter what specific columns are used in it:

MariaDB [test]> explain select count(t10.b+1) from t10, t11;
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
| id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | Extra                        |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
|    1 | SIMPLE      | NULL  | NULL | NULL          | NULL | NULL    | NULL | NULL | Select tables optimized away |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
1 row in set (0.001 sec)

MariaDB [test]> explain select count(t10.b+t11.b) from t10, t11;
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
| id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | Extra                        |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
|    1 | SIMPLE      | NULL  | NULL | NULL          | NULL | NULL    | NULL | NULL | Select tables optimized away |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
1 row in set (0.001 sec)

NULL-able expression disqualifies the optimization:

MariaDB [test]> explain select count(t10.b+t11.a) from t10, t11;
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+
| id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | Extra                              |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+
|    1 | SIMPLE      | t11   | ALL  | NULL          | NULL | NULL    | NULL | 3    |                                    |
|    1 | SIMPLE      | t10   | ALL  | NULL          | NULL | NULL    | NULL | 10   | Using join buffer (flat, BNL join) |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+
2 rows in set (0.001 sec)

But do we need to think about all these details now?

AFAIU what we need to do is to place hooks to save and/or infuse the record count in get_exact_record_count(). And let the optimizer code handle the rest as it currently does it

bsrikanth-mariadb · 2026-06-04T08:13:39Z

However, I see one problem. How could we get the table information if the expression inside the count() is not a field_type? For count(a) I could get table info from field a.

The optimizer already handles all this.

create table t10 (a int, b int not null) engine=myisam;
insert into t10 select seq,seq from seq_1_to_10;
create table t11 like t10;
insert into t11 select seq, seq from seq_1_to_3;

If the expression is not NULLable, it doesn't matter what specific columns are used in it:

MariaDB [test]> explain select count(t10.b+1) from t10, t11;
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
| id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | Extra                        |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
|    1 | SIMPLE      | NULL  | NULL | NULL          | NULL | NULL    | NULL | NULL | Select tables optimized away |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
1 row in set (0.001 sec)

MariaDB [test]> explain select count(t10.b+t11.b) from t10, t11;
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
| id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | Extra                        |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
|    1 | SIMPLE      | NULL  | NULL | NULL          | NULL | NULL    | NULL | NULL | Select tables optimized away |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------+
1 row in set (0.001 sec)

NULL-able expression disqualifies the optimization:

MariaDB [test]> explain select count(t10.b+t11.a) from t10, t11;
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+
| id   | select_type | table | type | possible_keys | key  | key_len | ref  | rows | Extra                              |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+
|    1 | SIMPLE      | t11   | ALL  | NULL          | NULL | NULL    | NULL | 3    |                                    |
|    1 | SIMPLE      | t10   | ALL  | NULL          | NULL | NULL    | NULL | 10   | Using join buffer (flat, BNL join) |
+------+-------------+-------+------+---------------+------+---------+------+------+------------------------------------+
2 rows in set (0.001 sec)

But do we need to think about all these details now?

AFAIU what we need to do is to place hooks to save and/or infuse the record count in get_exact_record_count(). And let the optimizer code handle the rest as it currently does it

okay. So, get_exact_record_count() wouldn't take any arguments such as TABLE*. Instead, depending on the invocation/call number we return the count. Is it?

During replay, when get_exact_record_count() is invoked from opt_sum_query(), hook the recorded table rows from the parsed context for each table, during the calculation of count.

bsrikanth-mariadb added the MariaDB Corporation label May 29, 2026

gemini-code-assist Bot reviewed May 29, 2026

View reviewed changes

bsrikanth-mariadb force-pushed the 13.1-MDEV-39791-handle-count-optimization-in-replay branch 2 times, most recently from 5f4ea85 to 65823a0 Compare May 29, 2026 09:16

spetrunia force-pushed the bb-12.3-MDEV-39368-test-replay branch from 039fe52 to 90b9f07 Compare May 29, 2026 11:57

bsrikanth-mariadb force-pushed the 13.1-MDEV-39791-handle-count-optimization-in-replay branch 5 times, most recently from 438acc1 to fabfa22 Compare June 2, 2026 23:24

spetrunia reviewed Jun 3, 2026

View reviewed changes

Comment thread sql/opt_sum.cc Outdated

spetrunia reviewed Jun 3, 2026

View reviewed changes

Comment thread mysql-test/main/opt_context_replay_basic.result Outdated

spetrunia reviewed Jun 3, 2026

View reviewed changes

Comment thread sql/opt_context_store_replay.cc Outdated

spetrunia reviewed Jun 3, 2026

View reviewed changes

Comment thread sql/opt_context_store_replay.cc Outdated

spetrunia requested changes Jun 3, 2026

View reviewed changes

bsrikanth-mariadb force-pushed the 13.1-MDEV-39791-handle-count-optimization-in-replay branch from fabfa22 to 80c607d Compare June 4, 2026 03:07

bsrikanth-mariadb force-pushed the 13.1-MDEV-39791-handle-count-optimization-in-replay branch 3 times, most recently from 1299232 to 5c9ac44 Compare June 4, 2026 11:34

spetrunia approved these changes Jun 4, 2026

View reviewed changes

MDEV-39791: Handle count aggregate optimization for replay purpose

cf18a5b

During replay, when get_exact_record_count() is invoked from opt_sum_query(), hook the recorded table rows from the parsed context for each table, during the calculation of count.

bsrikanth-mariadb force-pushed the 13.1-MDEV-39791-handle-count-optimization-in-replay branch from 5c9ac44 to cf18a5b Compare June 4, 2026 12:50

bsrikanth-mariadb merged commit cf18a5b into bb-12.3-MDEV-39368-test-replay Jun 4, 2026
1 of 3 checks passed

Uh oh!

Conversation

bsrikanth-mariadb commented May 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

spetrunia commented Jun 3, 2026

Uh oh!

spetrunia commented Jun 3, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

spetrunia left a comment

Choose a reason for hiding this comment

Uh oh!

bsrikanth-mariadb commented Jun 4, 2026

Uh oh!

bsrikanth-mariadb commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

spetrunia commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bsrikanth-mariadb commented Jun 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

bsrikanth-mariadb commented May 29, 2026 •

edited

Loading

bsrikanth-mariadb commented Jun 4, 2026 •

edited

Loading

spetrunia commented Jun 4, 2026 •

edited

Loading